Eecient Implementation of Reduce-scatter in Mpi

We discuss the eecient implementation of a collective operation called reduce-scatter , which is deened in the MPI standard. The reduce-scatter is equivalent to the combination of a reduction on vectors of length n with a scatter of the resulting n-vector to all processors. We describe the implementation issues and the performance characterization of two new algorithms for the reduce-scatter that have been proven to be highly eecient in theory under the assumption of fully connected parallel system. A performance comparison with existing mainstream implementations of the operation is presented which connrms the practical advantage of the new algorithms. Experiments show that the two algorithms have diierent characteristics which make them complementary in providing a performance gain over standard algorithms. Our study has been carried out in the context of the MPI standard on two diierent platforms: an SP2 and a Myrinet interconnected cluster of Pentium PRO. However, most of the results reported here are not speciic for either MPI or the platforms used, and they hold in general for any message passing programming system.